Optimization Framework with Minimum Description Length Principle for Probabilistic Programming
نویسندگان
چکیده
Application of the Minimum Description Length principle to optimization queries in probabilistic programming was investigated on the example of the C++ probabilistic programming library under development. It was shown that incorporation of this criterion is essential for optimization queries to behave similarly to more common queries performing sampling in accordance with posterior distributions and automatically implementing the Bayesian Occam’s razor. Experimental validation was conducted on the task of blood cell detection on microscopic images. Detection appeared to be possible using genetic programming query, and automatic penalization of candidate solution complexity allowed to choose the number of cells correctly avoiding overfitting.
منابع مشابه
Dynamic Programming Algorithm for Optimization of β-Decision Rules
We describe a dynamic programming algorithm that builds a decision rule for each row of a decision table. Rules are constructed according to the minimum description length principle: they have minimum length. Model complexity is regulated by adjusting rule accuracy.
متن کاملA New Minimum Description Length
The minimum description length(MDL) method is one of the pioneer methods of parametric order estimation with a wide range of applications. We investigate the definition of two-stage MDL for parametric linear model sets and exhibit some drawbacks of the theory behind the existing MDL. We introduce a new description length which is inspired by the Kolmogorov complexity principle.
متن کاملWaveform Design using Second Order Cone Programming in Radar Systems
Transmit waveform design is one of the most important problems in active sensing and communication systems. This problem, due to the complexity and non-convexity, has been always the main topic of many papers for the decades. However, still an optimal solution which guarantees a global minimum for this multi-variable optimization problem is not found. In this paper, we propose an attracting met...
متن کاملA Representational MDL Framework for Improving Learning Power of Neural Network Formalisms
Minimum description length (MDL) principle is one of the wellknown solutions for overlearning problem, specifically for artificial neural networks (ANNs). Its extension is called representational MDL (RMDL) principle and takes into account that models in machine learning are always constructed within some representation. In this paper, the optimization of ANNs formalisms as information represen...
متن کاملEfficient Optimization of an MDL-Inspired Objective Function for Unsupervised Part-Of-Speech Tagging
The Minimum Description Length (MDL) principle is a method for model selection that trades off between the explanation of the data by the model and the complexity of the model itself. Inspired by the MDL principle, we develop an objective function for generative models that captures the description of the data by the model (log-likelihood) and the description of the model (model size). We also ...
متن کامل